With the emergence of low cost high definition video cameras and high speed internet connectivity, video internet traffic is growing quickly. 360 degree video is an emerging trend in virtual reality, video conferencing, automotive, and surveillance applications, among others and is supported by wide field of view cameras, including 180 and 360 degree field of view cameras. Dedicated web sites and channels are also available to post and view 360 degree videos, including virtual reality content. For collaboration between groups of users, 360 degree video offers greater flexibility. Instead of switching between cameras, a remote viewer can move across a room using a 360 degree view and zoom in on people or objects of interest in the camera's view. Panoramic video playback systems using Virtual Reality (VR) head mounted displays (HMD) use high speed data connection and high speed processing in a HMD (Head Mounted Display) or other client device.
Panoramic or 360 degree video content is typically created by stitching together the 2D videos captured by multiple cameras into a spherical representation. In some cases, the multiple views are rendered in software. The spherical video is then projected into a format suited for video encoding, such as an equi-rectangular projection format or cube map projection format.
At the client, the bitstream containing the full panoramic compressed video frames is received and decoded. This creates a representation of the entire panoramic field of view, and a smaller viewport is selected for display. The particular portion of decoded video represented by the viewport is determined by the head position/orientation of an HMD, or through a position selection user interface when viewing on a normal 2D display.
A region from a panoramic video frame which contains the expected viewport to be viewed by a user may be extracted, projection mapped to a rectangular region in a rectilinear projection format, and re-encoded before the HMD receives the frame, in order to reduce the resolution required to be decoded at the client. This additional processing may be done in a MANE (Media Aware Network Element), or may be done in a computer that is connected to a HMD over a wireless network. The Media Aware Network Element signals information about the size and position of the extracted region, and the HMD uses this information to select the position of the viewport within this extracted region. The viewport is then used to determine which portion of the extracted region to display to the user.